Multi-camera vehicular vision system with graphic overlay

ABSTRACT

A vehicular vision system includes a plurality of cameras disposed at a vehicle and capturing image data as the equipped vehicle is driven along a traffic lane of a road. Captured image data is processed to detect a rearward-approaching vehicle and determine a path of travel of the detected rearward-approaching vehicle. Responsive to determining that the path of travel of the detected rearward-approaching vehicle is along a side traffic lane adjacent to the traffic lane along which the equipped vehicle is driven, the vehicular vision system (i) displays at a video display screen video images derived at least from image data captured by the side camera that is at the side portion of the equipped vehicle that is adjacent to the side traffic lane and (ii) overlays a transparent graphic overlay overlaying the displayed video images that is based on the determined path of travel of the detected rearward-approaching vehicle.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/948,474, filed Sep. 21, 2020, now U.S. Pat. No. 11,192,500,which is a continuation of U.S. patent application Ser. No. 16/693,515,filed Nov. 25, 2019, now U.S. Pat. No. 10,780,827, which is acontinuation of U.S. patent application Ser. No. 16/246,624, filed Jan.14, 2019, now U.S. Pat. No. 10,486,596, which is a continuation of U.S.patent application Ser. No. 14/191,512, filed Feb. 27, 2014, now U.S.Pat. No. 10,179,543, which claims the filing benefits of U.S.provisional applications, Ser. No. 61/793,592, filed Mar. 15, 2013, andSer. No. 61/770,051, filed Feb. 27, 2013, which are hereby incorporatedherein by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates generally to vision systems for vehiclesand, more particularly, to vision systems having a plurality ofexteriorly facing cameras disposed at a vehicle.

BACKGROUND OF THE INVENTION

Rear backup cameras and other exterior vision systems are known for usein vehicles. Examples of such systems are described in U.S. Pat. Nos.7,859,565; 6,611,202; 6,222,447; 5,949,331; 5,670,935 and/or 5,550,677,which are hereby incorporated herein by reference in their entireties.Such systems may display images for viewing by the driver of the vehiclethat provide a view exterior of the vehicle. It is known to provide aplurality of cameras at a vehicle, such as a forward facing camera, arearward facing camera and opposite sideward facing cameras, and tostitch together images captured by the cameras to provide a surroundview or top down view for displaying for viewing by a driver of thevehicle.

SUMMARY OF THE INVENTION

The present invention provides a means for dynamically calibrating oradjusting the image stitching of the images captured by two or morecameras of a multi-camera vision system of a vehicle, such as when theload or level of the vehicle changes.

The present invention provides a dynamic image stitching system orprocess that monitors overlapping regions of fields of view of adjacentcameras (such as a side camera and a front or rear camera) and adjuststhe image stitching of the captured images to adapt the image stitchingwhen the load of the vehicle changes or when the road level or terrainchanges or the like. A display screen displays the stitchedsubstantially seamless images for viewing by a driver of the vehiclewhen the driver is normally operating the vehicle.

Optionally, the dynamic image stitching system of the present inventionmay include an image processor that is operable to adjust a stitchingalgorithm responsive to a determination of a change in at least onecharacteristic of a feature or object present in an overlapping regionof the fields of view of two cameras at the vehicle (such as a sidecamera and a rear camera or such as a side camera and a front camera orthe like). For example, the processor may determine a feature or objectpresent in the overlapping region of the cameras' fields of view, andthe processor may compare a characteristic of the feature or object inimage data captured by a first camera with the correspondingcharacteristic of the feature or object in image data captured by asecond camera. The processor adjusts the stitching algorithm responsiveto a determination of a difference between a characteristic of a featureor object in image data captured by the first camera and thecharacteristic of the feature or object in image data captured by thesecond camera. For example, the processor may process captured imagedata to determine whether the feature as captured by the first camera ismisaligned with the feature as captured by the second camera, and theprocessor may adjust the stitching algorithm responsive to such adetermination.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle equipped with a multi-camera visionsystem in accordance with the present invention;

FIG. 2 is a top down view of a vehicle, with arrows indicating thegeneral fields of view of the cameras at the vehicle;

FIG. 3 shows the images captured by the cameras of the vehicle of FIG.2;

FIG. 4 is a plan view of a vehicle disposed at a calibration pattern;

FIG. 5 is an image captured by a side camera of a vehicle equipped withthe vision system of the present invention;

FIG. 6 is an image captured by a rear camera of a vehicle equipped withthe vision system of the present invention;

FIG. 7A is a schematic showing a vehicle with an ultrasound distancemeasuring system having an ultrasound sensor integrated into a sidemirror housing of the vehicle in accordance with the present invention,shown with the vehicle in a horizontal level;

FIG. 7B shows a vehicle similar to FIG. 7A, but with the vehicle rolledto its right by an angle ψ, with the measuring system that rolls withthe vehicle detecting a closer distance to objects and the ground;

FIG. 8 shows the image portions taken by the respective cameras, whichadd to a stitched virtual top view image, shown with objects symbolizedby boxes passing the scene surrounding the vehicle when the vehicle isdriving forward;

FIG. 9 shows a stitched virtual top view image derived from the imageportions, when the vehicle is leveled normally the objects are supposedto pass the scene in generally straight lines when the vehicle isdriving straight, shown with a picked out test object highlighted inblack;

FIG. 10 shows the identical scene as shown in FIG. 9, with two testobjects highlighted and other objects erased (for clearer illustration),showing that, when the vehicle is leveled normally, the test objects aresupposed to pass the scene in generally straight lines sideward or leftand right from the vehicle when the vehicle is driving straight;

FIG. 11 shows a scene similar to these in FIGS. 9 and 10, with two testobjects highlighted, with the vehicle driving straight as in FIGS. 9 and10, but with the vehicle rolled by an angle ψ to the right, when therolling angle ψ is uncompensated by the vision system, the passing pathsof the tracked (and also the untracked) objects do not follow straightlines and the stitching thresholds do not match any more;

FIGS. 12-14 are perspective views of a parking maneuvering situation(partially on the sidewalk), showing overlays in accordance with thepresent invention; and

FIGS. 15-17 are perspective views of a highway situation, showingoverlays in a view such as may be shown in a rearview side mirrorreplacement camera display in accordance with the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system or vision system 12that includes at least one exterior facing imaging sensor or camera,such as a rearward facing imaging sensor or camera 14 a (and the systemmay optionally include multiple exterior facing imaging sensors orcameras, such as a forwardly facing camera 14 b at the front (or at thewindshield) of the vehicle, and a sidewardly/rearwardly facing camera 14c, 14 d at respective sides of the vehicle), which captures imagesexterior of the vehicle, with the camera having a lens for focusingimages at or onto an imaging array or imaging plane or imager of thecamera (FIG. 1). The vision system 12 includes a control or processor 18that is operable to process image data captured by the cameras and mayprovide displayed images at a display device 16 for viewing by thedriver of the vehicle (although shown in FIG. 1 as being part of orincorporated in or at an interior rearview mirror assembly 20 of thevehicle, the control and/or the display device may be disposed elsewhereat or in the vehicle).

The dynamic top view vision system of the present invention stitchescaptured images to form a composite substantially seamless image, andmay utilize aspects of the vision systems described in U.S. Pat. Nos.6,222,447; 5,949,331; 5,760,962; 5,670,935 and/or 5,550,677, and/or U.S.patent applications, Ser. No. 12/508,840, filed Jul. 24, 2009, whichpublished on Jan. 28, 2010 as U.S. Publication No. US-2010-0020170; Ser.No. 13/848,796, filed Mar. 22, 2013, and published on Oct. 24, 2013 asU.S. Publication No. US-2013-0278769, Ser. No. 13/852,190, filed Mar.28, 2013, and published on Aug. 29, 2013 as U.S. Publication No.US-2013-0222593, Ser. No. 13/894,870, filed May 15, 2013, now U.S. Pat.No. 10,089,537, and/or Ser. No. 14/046,174, filed Oct. 4, 2013, now U.S.Pat. No. 9,723,272, which are hereby incorporated herein by reference intheir entireties.

The panoramic or surround view or top view or bird's eye view visionsystem of the present invention provides a stitched image that isderived from images or image data captured by a plurality of cameras atthe vehicle (such as the forward facing camera, the rearward facingcamera, the left side camera and the right side camera). In order toprovide a substantially seamless stitched image as displayed on a videodisplay screen viewable by the driver when normally operating theequipped vehicle, respective adjoining image portions of the capturedimages displayed as a composite image need to be properly aligned, sothat, for example, a road marking or parking line or similar feature orcharacteristic that is at least partially captured by one of the camerasand thus is displayed in its respective image portion, and that is alsoat least partially captured by a second one of the cameras and thus isdisplayed in its respective image portion, is properly aligned anddisplayed on the video screen as it should appear when the cameras areproperly calibrated and as it would appear in real life on the ground ator near the vehicle. By means of the present invention, misalignments,discontinuities and improperly joined lines in the stitched compositeimage that may be caused by the likes of vehicle loading and/or roadsurface out-of-flatness or the like, can be mitigated and preferably atleast substantially eliminated.

Uneven road conditions may make the stitched image look broken andmisaligned even with properly calibrated cameras. For example, one ormore wheels of the vehicle may be raised higher than the other wheels ofthe vehicle due to the road camber or tilt. Also, a change in vehicledynamics that differ from the vehicle dynamics at the time of theinitial multi-camera system calibration (when the cameras and visionsystem were initially calibrated, typically at the automobilemanufacturing plant itself) may make the stitched/composite displayedimage appear misaligned at the stitching edges/joints. Such a change invehicle dynamics may be due to uneven or changed load distribution atthe vehicle, such as a heavy load added at one side or at the rear ofthe vehicle or at the top of the vehicle causing the vehicle to tilttowards one side or rearward, or due to uneven passenger loading or thelike or uneven vehicle suspension performance or the like. Thus, thepresent invention provides a dynamic multi-camera top view or bird's eyeview or surround view system, which is operable to correct for suchrun-time/real-time errors or misalignments in the stitched images thatwould otherwise occur, and is operable to so do on an image-by-image orframe-by-frame basis in real time while the vehicle is being normallyoperated.

As described in U.S. patent application Ser. No. 13/847,815, filed Mar.20, 2013, and published on Oct. 31, 2013 as U.S. Publication No.US-2013-0286193, which is hereby incorporated herein by reference in itsentirety, a multi camera image processing system may be operable todetect the elevation of objects by its sheering effect when the objectis seen in an overlapping region by at least two of the vision systemfish eye lens cameras at different angles. The dynamic multi-cameravision system of the present invention also uses the overlapping regionsof the fields of view of the respective cameras of the system, such as,for example, the overlapping fields of view of a driver-side camera atan exterior sideview mirror of the vehicle and a front camera (with theoverlapping region being, for example, at the front left or driver-sidecorner of the vehicle). The system is operable to process image datarepresentative of overlapping regions in the adjacent camera fields ofview to detect corresponding features. When such corresponding featuresare detected or determined, the system may match the features.

For example, and with reference to FIGS. 5 and 6, the right side orpassenger side camera captures an image sideward of the vehicle, withthe captured image encompassing an area rearward and sideward of thevehicle (FIG. 5), while the rear camera captures an image rearward ofthe vehicle, with the captured image encompassing an area rearward andsideward of the vehicle (FIG. 6). The system is operable to detectcorresponding features in the imaged scene, such as the points shown inFIGS. 5 and 6, and may match the features so that, when the images arestitched, the matched corresponding features are generally orsubstantially aligned and displayed appropriately.

The system of the present invention is operable to generate affinetransforms or other suitable transforms that minimize the error infeature mapping from different views. Such an affine transformation oraffine map is a transformation that typically preserves straight lines(such that all points lying on a line initially still lie on a lineafter transformation) and that typically preserves ratios of distancesbetween points lying on a straight line and that typically preservessets of parallel lines so that they remain generally parallel aftertransformation.

The system of the present invention is further operable to apply thetransforms to correct the final stitched image. For example, the systemmay determine when there is a change in the characteristics detected ordetermined in the overlapping regions of the fields of view and mayadapt or adjust the image stitching algorithm or process to accommodatefor the detected or determined change. Because the system is a dynamicsystem, the system is operable to re-generate the transforms for everyfew frames to correct for dynamic conditions (such as changes in theroad conditions or changes in the vehicle loading or the like). Thepresent invention thus is operable to adapt the stitching of the imagesafter the cameras are calibrated to adapt the system and accommodatechanges in the vehicle and/or road surface that may otherwise adverselyaffect the quality or seamlessness of the stitched images. The presentinvention thus provides a vision system that maintains a substantiallyseamless stitched image for displaying for view by the driver of thevehicle through various driving and/or loading conditions of thevehicle.

The system thus determines differences in features present inoverlapping image data captured by a front (or rear) camera and inoverlapping image data captured by a side camera, and may adjust theimage stitching algorithm accordingly, such that the displayed stitchedimage provides an enhanced display (such as by assuring that image datacaptured by the front camera and image data captured by the side cameraappropriately align in the display viewed by the driver of the equippedvehicle so that there is no discontinuity or incongruity in the imagebeing displayed). A feature can be any one of a curb at the side of theroad being traveled by the equipped vehicle, a lane boundary marking atthe side of the road being traveled by the equipped vehicle, an edgeregion of the road being traveled by the equipped vehicle, a mailboxbeing approached and passed by the equipped vehicle, a pole or post orpillar at the side of the road being traveled by the equipped vehicle,and/or the like.

For example, while traveling forward on a road, the equipped vehicle mayapproach a mailbox located at or near a curb to the left of the equippedvehicle. The vehicle is equipped with a forward facing camera mounted atthe front of the vehicle and a side-mounted camera mounted at the leftor driver-side exterior rearview mirror of the vehicle. As the mailboxis being approached by the vehicle, it is first viewed/imaged by thefront or forward facing camera but is not yet viewed/imaged by the sidemounted camera. As the vehicle approaches closer to the mailbox, themailbox is viewed/imaged by both the forward facing camera and theside-mounted camera, and is located at a region or area where the fieldsof view of both cameras overlap and encompass the mailbox. In this fieldof view overlap region, the same object or feature (for example, themailbox) is commonly viewed by the forward facing camera and the sidemounted camera. However, the front camera's view/image of the object orfeature will be different from that of the side camera's view/image ofthe object or feature. In accordance with the present invention, thisdifference is determined by the image processor and the image stitchingalgorithm is adjusted according to the determined difference.

For example, if the mailbox is located in the overlap region of twocameras (such as at or near or along a stitching line or region of thecaptured images), the stitching algorithm is adjusted so that themailbox images (as captured by, for example, both the front camera andthe side camera) generally correspond to one another to provide agenerally smooth transition of the mailbox image as the mailbox displayimage moves or transitions from the front camera image to theoverlapping region and to the side camera image. For example, the imagestitching algorithm may be adjusted to adjust an orientation of themailbox so that the generally vertical post or structure of the mailboxin images derived from the image data captured by the front camera isgenerally aligned with the generally vertical post or structure of themailbox in images derived from the image data captured by the sidecamera. The system thus accommodates for misalignment of characteristicsof features or objects between the images of the front camera and sidecamera (or the rear camera and side camera).

Therefore, the present invention provides a dynamic process forautomatically stitching images captured by the cameras of a multi-cameravision system of a vehicle, such as for a top-down or bird's eye viewdisplay for viewing by the driver of the vehicle. For example, thepresent invention provides a system that adjusts the stitching ofadjacent images captured by the front, rear and driver side andpassenger side cameras for a surround view or top-down view visionsystem of a vehicle.

The present invention utilizes the fact that an object that is locatedin the overlapping region of the fields of view of two of the cameras ofthe multi-camera system (such as, for example, a driver-side camera anda rear camera of the vehicle) is commonly imaged by that side camera andthat rear camera. However, because of the different and known differentplacement or positions of the cameras at the equipped vehicle, theobject will be viewed differently by the respective first and secondcamera. Such differences are accommodated for and are factored in viathe initial factory calibration so that the system displays the objectappropriately and correctly in the stitched composite image. However,if, for example, during dynamic loading of the vehicle, these respectiveviews differ from what should occur in an unloaded or unperturbedideally calibrated scenario, this determined change or difference can beutilized using image processing/algorithmic techniques to adjust theimage stitching to compensate for the dynamic loading of the vehicle inreal time. Thus, utilizing the present invention, the stitching of thecaptured images (to provide or generate the stitched composite image)may be adapted for vehicle load changes and level changes and the likein real time and during normal operation of the vehicle.

Optionally, when the vehicle is in motion (such as driving forwardly orrearwardly), the ground level calibration may be checked permanently orepisodically or from time to time, or preferably at times when thesystem may have unused processing resources. As shown in FIGS. 8-11,image portions 14 a′, 14 b′, 14 c′ and 14 d′ taken by the respectivecameras 14 a, 14 b, 14 c and 14 d, and are used to create to a stitchedvirtual top view image. In FIG. 8, objects 30 are shown symbolized byboxes passing the scene surrounding the vehicle when the vehicle isdriving forward. A stitched virtual top view image may be derived fromthe image portions 14 a′, 14 b′, 14 c′ and 14 d′. When the vehicle isleveled normally, the objects 30 are supposed to pass the scene ingenerally straight lines when the vehicle is driving straight, shown inFIG. 9 with a picked out test object 31 highlighted in black. FIG. 10shows the identical scene as shown in FIG. 9, with two test objects 31highlighted and other objects erased (for clearer illustration), showingthat, when the vehicle is leveled normally, the test objects 31 aresupposed to pass the scene in generally straight lines sideward or leftand right from the vehicle when the vehicle is driving straight. FIG. 11shows a scene similar to these in FIGS. 9 and 10, with two test objects31 highlighted, with the vehicle driving straight as in FIGS. 9 and 10,but with the vehicle rolled by an angle ψ to the right, when the rollingangle ψ is uncompensated by the vision system, the passing paths of thetracked (and also the untracked) objects do not follow straight linesand the stitching thresholds 33 do not match any more.

The system of the present invention may track objects (such as objects30 and tracked objects 31 shown in FIGS. 8-11) detected coming into thedetection range in front of the vehicle (or to the rear of the vehiclewhen the vehicle is backing up) and may continue to track these objectswhen the vehicle is passing the objects and the objects are entering theviewing areas of different cameras (14 a to 14 d) of the vehicle, suchas can be seen with reference to FIGS. 8 and 9. The passing way of thevehicle is known and consistent relative to immobile objects. When anobject 31 (tracked) is entering the viewing area (14 d′ or 14 c′) of aside camera (14 d or 14 c), with the object being earlier tracked by thefront camera 14 b (while passing), the entering point (32 in FIG. 10)and passing way of the object through the side camera image isexpectable or predictable.

When there is an offset 33 or a different angle, the side camera will beoff by a different angle which points to mis-calibration of the camerasrelative to each other (in roll, yaw, pitch angle or transversaldisplacement) or the vehicle itself is skewed. Typically, both cases canbe separated or distinguished by the fact that skewing of the vehiclemay happen temporarily while a camera displacement stays in allconditions. Preferably, the system of the present invention may becontrolled in a way that first the camera alignment relative to anothercamera is calibrated and from then on skewing of the vehicle may beassumed when object path inconsistencies appear. Another decisioncriteria may be that object path inconsistencies may appear (in major)at all four cameras at the same time (such as shown in FIG. 11) during acamera displacement of one camera occurrences inconsistencies withinthree stitched camera images. When comparing FIG. 10, which shows thecase of a leveled vehicle, to FIG. 11, which shows the case of a skewedvehicle (rolled to the right in x axis), the passing ways of objectspassing the vehicle on the left side have an offset into the samedirection (to the right) as objects passing the vehicle at the rightside. Responsive to determination of such skewing or rolling of thevehicle, the system may alter the virtual ground plane in a way that avirtual top view as like in FIG. 10 results.

Additionally or optionally, the system may also take an input from anenvironmental sensor 28 (beside the cameras) into account. If not doneby the vision computing system alone, the system may utilize distancemeasuring sensors, such as, for example, ultrasound (exemplary case inFIGS. 7A and 7B), Time of Flight (TOF), RADAR, LIDAR, LADAR and/or thelike, for detecting the true ground plane level and elevations 25 nearby the vehicle 10 (the detection range which may have (still) adequateresolution may be from about 0.2 m up to about 10 m depending on thesensors type (equates to ultrasound cloud 29 in FIGS. 7A and 7B)) forfeeding it into the vision system to correct the virtual (assumed)ground plane for the virtual top view vision projection in the mannerthe vehicle is rolled or pitched. The system either may detect changesfor adding/subtracting delta tunings or absolute values for plain input.

For example, and as shown in FIGS. 7A and 7B, the vehicle 10 may includean ultrasound distance measuring system having an ultrasound sensor 28integrated into a side mirror housing 27 of the vehicle 10. Theultrasound sensor 28 is shown with an effective ultrasound expansioncloud illustration 29 (not actually visible), and shown with the vehiclein a horizontal level. If the vehicle is rolled or tilted to its rightby an angle ψ, the measuring system, which rolls with the vehicle,detects a closer distance to objects and the ground. FIG. 7B shows avehicle turned by an angle ψ (rolled in x axis) in comparison to aleveled car in FIG. 7A. The system may detect the ground or at leastelevated objects on the ground 25 are coming closer to the mirrorintegrated sensor due to that rolling of the vehicle by the angle ψ. Thesensor data may come as a depth image or in a point cloud which may needto be combined with the image giving (camera) system image data bysensor fusion. The ground plane, objects (with elevation) and horizonmay be distinguishable by processing of sensor data.

As an additional or alternative way to detect that the vehicle is notfully leveled horizontally (rolled in x axis) and/or has a pitch angle(in y axis) or the like, the system may utilize the compression ratio ofthe shock absorber system or suspension system of all wheels shockabsorbers of the vehicle. Nowadays vehicles usually have sensors todetect the compression rate and usually this signal is provided via thevehicle's CAN or vehicle bus network, which makes it possible that thevision system is able to process this information. By such processing ofsuspension information (for the individual wheels of the vehicle) thesystem may determine uneven suspension of the corner regions of thevehicle and may correct the virtual (assumed) ground plane for thevirtual top view vision projection in accordance with the determinedratio that the vehicle is rolled or pitched (skewed).

As a preferred embodiment, the system of the present invention maydetermine the ground level calibration roughly on hand of the shockabsorber sensor or suspension data (via CAN) and may use the opticalcalibration referred to above as a successive fine tuning procedure,which may use the optional range sensor data in addition or in backup.

It is desirable for vision overlays to look quite artificial when thesebecome mapped over a real scene view captured by automotive cameras.Earlier filings, such as International Publication No. WO 2013/086249,which is hereby incorporated herein by reference in its entirety, usevisual effects such as ‘motion blurring’ and/or partially transparencyin dependence of the viewing distance and direction of the virtualcamera to help this fact. Nevertheless there is room for improvement.

As soon as the 3D shape or topography of the world within a displayedscene (in an automotive vision system) is known (using a mono camera and3D recognition by ‘structure from motion’ or ‘structure from focallength’ or ‘depths estimation from focal blur’ (known from known TV 2Dto 3D conversion algorithms (TP-Vision in Philips TV) or by using stereocameras (including infrared) or using a mono cam era and additionally asecond or non-vision-based sensor system, such as LIDAR, RADAR, TOF orthe like), especially the ground surface structure the car is restingon, will be renderable. The overlays of the present invention may bemapped or placed (or calculated or controlled or adjusted) responsive tothe world's topographical structure. Such overlays may utilize aspectsof the overlays described in International Publication No. WO2009036176, which is hereby incorporated herein by reference in itsentirety.

Overlays related to the subject or equipped or host vehicle and overlaysdedicated to other vehicles, infrastructure or points of interest (POI)may be mapped onto the structures' topography as like projected ordropped from over top. By that the overlays cover the mapping structurefolding around all ups and downs which strengthens the three dimensionalimpression of the scenery (see FIG. 12 (flat overlay)), since the top ofthe overlay may follow to the topography.

The overlays may have a three dimensional structure which top elevationmay be responsive to the ground plane (virtually). By that the top ofthe overlay repeats the topography of its ground plane (See FIG. 13(solid overlay)).

The two and three dimensional overlays may be fully or partiallytransparent for preventing anything important in the scene is hidden byan overlay (and not detected or as important classified). See FIG. 14(transparent overlay).

FIGS. 12, 13 and 14 have in common a scene of a pedestrian side walkalong a road. The displayed view, which may be displayed at a vehicle'scompartment display when the vehicle is reversing into a long sideparking spot, is indicated by the box or outline. As common, thecurvature of the reversing path according to the steering wheel angle orthe automated path planning system is indicated by the line of(positions of) overlays. The overlays may have different colorsaccording to the distance to the vehicle's rear. The overlays may havedifferent colors or may change color or blink or flash or may alter intransparency or in brightness or may be animated, such as by torsion,wobbling, vibrating or jumping (such as in a manner like jelly or thelike), responsive to the distance to the vehicle's rear, the speed atwhich possible colliding hazards or possible damage threatening objectsmay close to the vehicle (or that the vehicle may close to the objects),or the remaining (eventually estimated, by the trajectory in past) timeuntil closing to an object or the like. In all three of FIGS. 12-14, theexamples of the vehicle's driving path lead or progress substantially ormassively up the curb at the side of the road and over the side walk.The curbs may be visible by the step in heights of the overlay's top andoptionally additionally by a thread indication or highlighting asdescribed above (such as by a change in color, blinking, brightness oranimation or the like).

With reference now to FIG. 15, the system of the present invention mayvisualize that the arrangement of the overlays may reflect the threedimensional topography of a scene. The shown perspective views of afreeway situation, such as may be displayed in a rearview mirrorreplacement camera display (such as by utilizing aspects of the systemsdescribed in U.S. provisional application Ser. No. 61/935,057, filedFeb. 3, 2014, which has now been filed as U.S. patent application Ser.No. 14/898,493, filed Dec. 15, 2015, and published on May 19, 2016 asU.S. Publication No. US-2016-0137126), may be used for visualizing ordisplaying the projected path of a vehicle driving behind the subject orequipped vehicle but in another lane. Due to the overlayed boxes (whichmay have different highlightings as described above) following theprospective projected path of the following vehicle, the equippedvehicle's driver may be able to readily capture at which specific lanethe following vehicle is approaching.

Not shown in the drawings is that when the overlays infringe (take thesame space) another (real or virtual) object, the overlays may appear ina faded color or in dot or dashed or phantom lines. Optionally, theobjects may pile on top of each other (when running across a wall forexample). Optionally, the overlays may shrink in size or step slightlyto the side when logically possible. For example, an inserted ‘Point ofInterest Label’ may give way to a parking overlay which position issystematically crucial.

It is also desirable to gather and provide context depending labeling ofinformation in vehicle vision systems. Storing a lot of information onboard the vehicle takes a lot of storage space and it may becomeoutdated comparably fast. Solutions are known to provide usefulinformation to the car aid systems and to the driver via wireless radiochannels from and to a dedicated server or from the world wide web, suchas by utilizing aspects of the systems described in U.S. patentapplication Ser. No. 13/660,306, filed Oct. 25, 2012, now U.S. Pat. No.9,146,898, which is hereby incorporated herein by reference in itsentirety. New standards are coming up as like HTMLS. This is forproviding displaying contents. To maintain the displaying contents up todate adds costs to the system. These costs may be fully or partiallyborne by third party providers. These may finance their services byadding commercial contents to the ‘useful contents’. For example, theseproviders may employ a service for ‘points of interest’ (POI) which maycome up as virtual labels added to real camera taken scenes of an urbanstreet or environment. The providers may collect money from stores andthe company or store logos or labels may be presented in a preferred orselected manner (such as in the foreground, brighter, bigger, moreexposed, blinking, or the like).

The vision system of the present invention may employ a spam filter forincoming foreign viewing contents. The spam filter properties may beconfigurable. The configuration may come from third parties or may bedone by the vision system provider or by the vehicle OEM or by thevehicle owner itself (over an HMI) or from a remote device or system(such as, for example, a cell phone app or the like). The spam filterproperties may be context dependent (situational, position dependent,vehicle state dependent, vehicle occupation or occupant dependent,driving state dependent). For example there may be contents preferred bythe adult occupants, while other or additional contents may becomedisplayed only when a teenager is also on board the vehicle. Forexample, the teenager may love fast food restaurants while the parentsprefer exquisite restaurants. Thus, the vision display system may showPOIs of all types of restaurants when the children are in the vehiclewith their parents, but will show just some exquisite flaggedrestaurants when the parents are alone in the vehicle. The “flagging”may come from the vision system provider, by the vehicle OEM or by thevehicle owner itself (over an HMI) or from remote (such as, for example,a cell phone app). Besides that all other annoying content may beoptionally suppressible by a spam filter.

Optionally, there may be different spam filter property data sets thatthe vehicle users are able to store in the vehicle data storage on theircell phone or remote. It may be possible that the spam filter properties(and all other vision system settings) may stay with the driver whichever vehicle he or she uses. The data set may be downloaded from aremote location (server) via any data channel or transferred by his/hercell phone.

The vision system may collect POIs by itself such as, for example, byemploying an object detection algorithm on known labels or by requestingthe background content according to detected maybe unknown labels. Thefound POIs may be distributed in a database common to all or tocertified users.

Optionally, the vision system (utilizing the forward facing camera and arearward facing camera and other cameras disposed at the vehicle withexterior fields of view) may be part of or may provide a display of atop-down view or birds-eye view system of the vehicle or a surround viewat the vehicle, such as by utilizing aspects of the vision systemsdescribed in International Publication Nos. WO 2010/099416; WO2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869,and/or U.S. patent application Ser. No. 13/333,337, filed Dec. 21, 2011,now U.S. Pat. No. 9,264,672, which are hereby incorporated herein byreference in their entireties.

The system includes an image processor operable to process image datacaptured by the camera or cameras, such as for detecting objects orother vehicles or pedestrians or the like in the field of view of one ormore of the cameras. For example, the image processor may comprise anEYEQ2 or EYEQ3 image processing chip available from Mobileye VisionTechnologies Ltd. of Jerusalem, Israel, and may include object detectionsoftware (such as the types described in U.S. Pat. Nos. 7,855,755;7,720,580 and/or 7,038,577, which are hereby incorporated herein byreference in their entireties), and may analyze image data to detectvehicles and/or other objects. Responsive to such image processing, andwhen an object or other vehicle is detected, the system may generate analert to the driver of the vehicle and/or may generate an overlay at thedisplayed image to highlight or enhance display of the detected objector vehicle, in order to enhance the driver's awareness of the detectedobject or vehicle or hazardous condition during a driving maneuver ofthe equipped vehicle.

The vehicle may include any type of sensor or sensors, such as imagingsensors or radar sensors or lidar sensors or ladar sensors or ultrasonicsensors or the like. The imaging sensor or camera may capture image datafor image processing and may comprise any suitable camera or sensingdevice, such as, for example, an array of a plurality of photosensorelements arranged in at least 640 columns and 480 rows (preferably amegapixel imaging array or the like), with a respective lens focusingimages onto respective portions of the array. The photosensor array maycomprise a plurality of photosensor elements arranged in a photosensorarray having rows and columns. The logic and control circuit of theimaging sensor may function in any known manner, and the imageprocessing and algorithmic processing may comprise any suitable meansfor processing the images and/or image data.

For example, the vision system and/or processing and/or camera and/orcircuitry may utilize aspects described in U.S. Pat. Nos. 7,005,974;5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545;6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268;6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563;6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258; 7,145,519;7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466; 7,592,928;7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or 5,786,772,and/or International Publication Nos. WO 2011/028686; WO 2010/099416; WO2012/061567; WO 2012/068331; WO 2012/075250; WO 2012/103193; WO2012/0116043; WO 2012/0145313; WO 2012/0145501; WO 2012/145818; WO2012/145822; WO 2012/158167; WO 2012/075250; WO 2012/0116043; WO2012/0145501; WO 2012/154919; WO 2013/019707; WO 2013/016409; WO2013/019795; WO 2013/067083; WO 2013/070539; WO 2013/043661; WO2013/048994; WO 2013/063014, WO 2013/081984; WO 2013/081985; WO2013/074604; WO 2013/086249; WO 2013/103548; WO 2013/109869; WO2013/123161; WO 2013/126715; WO 2013/043661 and/or WO 2013/158592 and/orU.S. patent applications, Ser. No. 14/163,325, filed Jan. 24, 2014, andpublished on Jul. 31, 2014 as U.S. Publication No. US-2014-0211009; Ser.No. 14/159,772, filed Jan. 21, 2014, now U.S. Pat. No. 9,068,390; Ser.No. 14/107,624, filed Dec. 16, 2013, now U.S. Pat. No. 9,140,789; Ser.No. 14/102,981, filed Dec. 11, 2013, now U.S. Pat. No. 9,558,409; Ser.No. 14/102,980, filed Dec. 11, 2013, and published Jun. 19, 2014 as U.S.Publication No. US-2014-0168437; Ser. No. 14/098,817, filed Dec. 6,2013, and published Jun. 19, 2014 as U.S. Publication No.US-2014-0168415; Ser. No. 14/097,581, filed Dec. 5, 2013, now U.S. Pat.No. 9,481,301; Ser. No. 14/093,981, filed Dec. 2, 2013, now U.S. Pat.No. 8,917,169; Ser. No. 14/093,980, filed Dec. 2, 2013, now U.S. Pat.No. 10,025,994; Ser. No. 14/082,573, filed Nov. 18, 2013, now U.S. Pat.No. 9,743,002; Ser. No. 14/082,574, filed Nov. 18, 2013, now U.S. Pat.No. 9,307,640; Ser. No. 14/082,575, filed Nov. 18, 2013, now U.S. Pat.No. 9,090,234; Ser. No. 14/082,577, filed Nov. 18, 2013, now U.S. Pat.No. 8,818,042; Ser. No. 14/071,086, filed Nov. 4, 2013, now U.S. Pat.No. 8,886,401; Ser. No. 14/076,524, filed Nov. 11, 2013, now U.S. Pat.No. 9,077,962; Ser. No. 14/052,945, filed Oct. 14, 2013, now U.S. Pat.No. 9,707,896; Ser. No. 14/046,174, filed Oct. 4, 2013, now U.S. Pat.No. 9,707,896; Ser. No. 14/016,790, filed Oct. 3, 2013, now U.S. Pat.No. 9,761,142; Ser. No. 14/036,723, filed Sep. 25, 2013, now U.S. Pat.No. 9,446,713; Ser. No. 14/016,790, filed Sep. 3, 2013, now U.S. Pat.No. 9,761,142; Ser. No. 14/001,272, filed Aug. 23, 2013, now U.S. Pat.No. 9,233,641; Ser. No. 13/970,868, filed Aug. 20, 2013, now U.S. Pat.No. 9,365,162; Ser. No. 13/964,134, filed Aug. 12, 2013, now U.S. Pat.No. 9,340,227; Ser. No. 13/942,758, filed Jul. 16, 2013, and publishedon Jan. 23, 2014 as U.S. Publication No. US-2014-0025240; Ser. No.13/942,753, filed Jul. 16, 2013, and published Jan. 30, 2014 as U.S.Publication No. US-2014-0028852; Ser. No. 13/927,680, filed Jun. 26,2013, and published Jan. 2, 2014 as U.S. Publication No.US-2014-0005907; Ser. No. 13/916,051, filed Jun. 12, 2013, now U.S. Pat.No. 9,077,098; Ser. No. 13/894,870, filed May 15, 2013, now U.S. Pat.No. 10,089,537; Ser. No. 13/887,724, filed May 6, 2013, now U.S. Pat.No. 9,670,895; Ser. No. 13/852,190, filed Mar. 28, 2013, and publishedAug. 29, 2013 as U.S. Publication No. US-2013-0222593; Ser. No.13/851,378, filed Mar. 27, 2013, now U.S. Pat. No. 9,319,637; Ser. No.13/848,796, filed Mar. 22, 2012, and published Oct. 24, 2013 as U.S.Publication No. US-2013-0278769; Ser. No. 13/847,815, filed Mar. 20,2013, and published Oct. 31, 2013 as U.S. Publication No.US-2013-0286193; Ser. No. 13/800,697, filed Mar. 13, 2013, and publishedOct. 3, 2013 as U.S. Publication No. US-2013-0258077; Ser. No.13/785,099, filed Mar. 5, 2013, now U.S. Pat. No. 9,565,342; Ser. No.13/779,881, filed Feb. 28, 2013, now U.S. Pat. No. 8,694,224; Ser. No.13/774,317, filed Feb. 22, 2013, now U.S. Pat. No. 9,269,263; Ser. No.13/774,315, filed Feb. 22, 2013, and published Aug. 22, 2013 as U.S.Publication No. US-2013-0215271; Ser. No. 13/681,963, filed Nov. 20,2012, now U.S. Pat. No. 9,264,673; Ser. No. 13/660,306, filed Oct. 25,2012, now U.S. Pat. No. 9,146,898; Ser. No. 13/653,577, filed Oct. 17,2012, now U.S. Pat. No. 9,174,574; and/or Ser. No. 13/534,657, filedJun. 27, 2012, and published Jan. 3, 2013 as U.S. Publication No.US-2013-0002873, and/or U.S. provisional applications, Ser. 61/931,811,filed Jan. 27, 2014; Ser. No. 61/919,129, filed Dec. 20, 2013; Ser. No.61/919,130, filed Dec. 20, 2013; Ser. No. 61/919,131, filed Dec. 20,2013; Ser. No. 61/919,147, filed Dec. 20, 2013; Ser. No. 61/919,138,filed Dec. 20, 2013, Ser. No. 61/919,133, filed Dec. 20, 2013; Ser. No.61/918,290, filed Dec. 19, 2013; Ser. No. 61/915,218, filed Dec. 12,2013; Ser. No. 61/912,146, filed Dec. 5, 2013; Ser. No. 61/911,666,filed Dec. 4, 2013; Ser. No. 61/911,665, filed Dec. 4, 2013; Ser. No.61/905,461, filed Nov. 18, 2013; Ser. No. 61/905,462, filed Nov. 18,2013; Ser. No. 61/901,127, filed Nov. 7, 2013; Ser. No. 61/895,610,filed Oct. 25, 2013; Ser. No. 61/895,609, filed Oct. 25, 2013; Ser. No.61/893,489, filed Oct. 21, 2013; Ser. No. 61/886,883, filed Oct. 4,2013; Ser. No. 61/879,837, filed Sep. 19, 2013; Ser. No. 61/879,835,filed Sep. 19, 2013; Ser. No. 61/878,877, filed Sep. 17, 2013; Ser. No.61/875,351, filed Sep. 9, 2013; Ser. No. 61/869,195, filed. Aug. 23,2013; Ser. No. 61/864,835, filed Aug. 12, 2013; Ser. No. 61/864,836,filed Aug. 12, 2013; Ser. No. 61/864,837, filed Aug. 12, 2013; Ser. No.61/864,838, filed Aug. 12, 2013; Ser. No. 61/856,843, filed Jul. 22,2013, Ser. No. 61/845,061, filed Jul. 11, 2013; Ser. No. 61/844,630,filed Jul. 10, 2013; Ser. No. 61/844,173, filed Jul. 9, 2013; Ser. No.61/844,171, filed Jul. 9, 2013; Ser. No. 61/842,644, filed Jul. 3, 2013;Ser. No. 61/840,542, filed Jun. 28, 2013; Ser. No. 61/838,619, filedJun. 24, 2013; Ser. No. 61/838,621, filed Jun. 24, 2013; Ser. No.61/837,955, filed Jun. 21, 2013; Ser. No. 61/836,900, filed Jun. 19,2013; Ser. No. 61/836,380, filed Jun. 18, 2013; Ser. No. 61/834,129,filed Jun. 12, 2013; Ser. No. 61/833,080, filed Jun. 10, 2013; Ser. No.61/830,375, filed Jun. 3, 2013; Ser. No. 61/830,377, filed Jun. 3, 2013;Ser. No. 61/825,752, filed May 21, 2013; Ser. No. 61/825,753, filed May21, 2013; Ser. No. 61/823,648, filed May 15, 2013; Ser. No. 61/823,644,filed May 15, 2013; Ser. No. 61/821,922, filed May 10, 2013; Ser. No.61/819,835, filed May 6, 2013; Ser. No. 61/819,033, filed May 3, 2013;Ser. No. 61/816,956, filed Apr. 29, 2013; Ser. No. 61/815,044, filedApr. 23, 2013; Ser. No. 61/814,533, filed Apr. 22, 2013; Ser. No.61/813,361, filed Apr. 18, 2013; Ser. No. 61/810,407, filed Apr. 10,2013; Ser. No. 61/808,930, filed Apr. 5, 2013; Ser. No. 61/806,674,filed Mar. 29, 2013; Ser. No. 61/793,592, filed Mar. 15, 2013; Ser. No.61/772,015, filed Mar. 4, 2013; Ser. No. 61/772,014, filed Mar. 4, 2013;Ser. No. 61/770,051, filed Feb. 27, 2013; Ser. No. 61/766,883, filedFeb. 20, 2013; Ser. No. 61/760,366, filed Feb. 4, 2013, and/or Ser. No.61/760,364, filed Feb. 4, 2013, which are all hereby incorporated hereinby reference in their entireties. The system may communicate with othercommunication systems via any suitable means, such as by utilizingaspects of the systems described in International Publication Nos. WO2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S. patentapplication Ser. No. 13/202,005, filed Aug. 17, 2011, now U.S. Pat. No.9,126,525, which are hereby incorporated herein by reference in theirentireties.

The imaging device and control and image processor and any associatedillumination source, if applicable, may comprise any suitablecomponents, and may utilize aspects of the cameras and vision systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,877,897; 6,498,620; 5,670,935;5,796,094; 6,396,397; 6,806,452; 6,690,268; 7,005,974; 7,937,667;7,123,168; 7,004,606; 6,946,978; 7,038,577; 6,353,392; 6,320,176;6,313,454 and 6,824,281, and/or International Publication Nos. WO2010/099416; WO 2011/028686 and/or WO 2013/016409, and/or U.S. Pat.Publication No. US 2010-0020170, and/or U.S. patent application Ser. No.13/534,657, filed Jun. 27, 2012, and published Jan. 3, 2013 as U.S.Publication No. US-2013-0002873, which are all hereby incorporatedherein by reference in their entireties. The camera or cameras maycomprise any suitable cameras or imaging sensors or camera modules, andmay utilize aspects of the cameras or sensors described in U.S. Pat.Publication No. US-2009-0244361 and/or U.S. patent application Ser. No.13/260,400, filed Sep. 26, 2011, now U.S. Pat. No. 8,542,451, and/orU.S. Pat. Nos. 7,965,336 and/or 7,480,149, which are hereby incorporatedherein by reference in their entireties. The imaging array sensor maycomprise any suitable sensor, and may utilize various imaging sensors orimaging array sensors or cameras or the like, such as a CMOS imagingarray sensor, a CCD sensor or other sensors or the like, such as thetypes described in U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962;5,715,093; 5,877,897; 6,922,292; 6,757,109; 6,717,610; 6,590,719;6,201,642; 6,498,620; 5,796,094; 6,097,023; 6,320,176; 6,559,435;6,831,261; 6,806,452; 6,396,397; 6,822,563; 6,946,978; 7,339,149;7,038,577; 7,004,606; 7,720,580 and/or 7,965,336, and/or InternationalPublication Nos. WO 2009/036176 and/or WO 2009/046268, which are allhereby incorporated herein by reference in their entireties.

The camera module and circuit chip or board and imaging sensor may beimplemented and operated in connection with various vehicularvision-based systems, and/or may be operable utilizing the principles ofsuch other vehicular systems, such as a vehicle headlamp control system,such as the type disclosed in U.S. Pat. Nos. 5,796,094; 6,097,023;6,320,176; 6,559,435; 6,831,261; 7,004,606; 7,339,149 and/or 7,526,103,which are all hereby incorporated herein by reference in theirentireties, a rain sensor, such as the types disclosed in commonlyassigned U.S. Pat. Nos. 6,353,392; 6,313,454; 6,320,176 and/or7,480,149, which are hereby incorporated herein by reference in theirentireties, a vehicle vision system, such as a forwardly, sidewardly orrearwardly directed vehicle vision system utilizing principles disclosedin U.S. Pat. Nos. 5,550,677; 5,670,935; 5,760,962; 5,877,897; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978 and/or 7,859,565, which are all herebyincorporated herein by reference in their entireties, a trailer hitchingaid or tow check system, such as the type disclosed in U.S. Pat. No.7,005,974, which is hereby incorporated herein by reference in itsentirety, a reverse or sideward imaging system, such as for a lanechange assistance system or lane departure warning system or for a blindspot or object detection system, such as imaging or detection systems ofthe types disclosed in U.S. Pat. Nos. 7,720,580; 7,038,577; 5,929,786and/or 5,786,772, and/or U.S. patent applications, Ser. No. 11/239,980,filed Sep. 30, 2005, now U.S. Pat. No. 7,881,496, and/or U.S.provisional applications, Ser. No. 60/628,709, filed Nov. 17, 2004; Ser.No. 60/614,644, filed Sep. 30, 2004; Ser. No. 60/618,686, filed Oct. 14,2004; Ser. No. 60/638,687, filed Dec. 23, 2004, which are herebyincorporated herein by reference in their entireties, a video device forinternal cabin surveillance and/or video telephone function, such asdisclosed in U.S. Pat. Nos. 5,760,962; 5,877,897; 6,690,268 and/or7,370,983, and/or U.S. Publication No. US-2006-0050018, which are herebyincorporated herein by reference in their entireties, a traffic signrecognition system, a system for determining a distance to a leading ortrailing vehicle or object, such as a system utilizing the principlesdisclosed in U.S. Pat. Nos. 6,396,397 and/or 7,123,168, which are herebyincorporated herein by reference in their entireties, and/or the like.

Optionally, the circuit board or chip may include circuitry for theimaging array sensor and or other electronic accessories or features,such as by utilizing compass-on-a-chip or EC driver-on-a-chip technologyand aspects such as described in U.S. Pat. Nos. 7,255,451 and/or7,480,149; and/or U.S. patent applications, Ser. No. 11/226,628, filedSep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008, and/or Ser. No. 12/578,732, filed Oct. 14, 2009, nowU.S. Pat. No. 9,487,144, which are hereby incorporated herein byreference in their entireties.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, for example, the vision system may include a videodisplay device disposed at or in the interior rearview mirror assemblyof the vehicle, such as by utilizing aspects of the video mirror displaysystems described in U.S. Pat. No. 6,690,268 and/or U.S. patentapplication Ser. No. 13/333,337, filed Dec. 21, 2011, now U.S. Pat. No.9,264,672, which are hereby incorporated herein by reference in theirentireties. The video mirror display may comprise any suitable devicesand systems and optionally may utilize aspects of the compass displaysystems described in U.S. Pat. Nos. 7,370,983; 7,329,013; 7,308,341;7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305;5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226; 5,802,727;5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252 and/or 6,642,851,and/or European patent application, published Oct. 11, 2000 underPublication No. EP 0 1043566, and/or U.S. patent application Ser. No.11/226,628, filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S.Publication No. US-2006-0061008, which are all hereby incorporatedherein by reference in their entireties. Optionally, the video mirrordisplay screen or device may be operable to display images captured by arearward viewing camera of the vehicle during a reversing maneuver ofthe vehicle (such as responsive to the vehicle gear actuator beingplaced in a reverse gear position or the like) to assist the driver inbacking up the vehicle, and optionally may be operable to display thecompass heading or directional heading character or icon when thevehicle is not undertaking a reversing maneuver, such as when thevehicle is being driven in a forward direction along a road (such as byutilizing aspects of the display system described in PCT Application No.PCT/US2011/056295, filed Oct. 14, 2011 and published Apr. 19, 2012 asInternational Publication No. WO 2012/051500, which is herebyincorporated herein by reference in its entirety).

Optionally, a video mirror display may be disposed rearward of andbehind the reflective element assembly and may comprise a display suchas the types disclosed in U.S. Pat. Nos. 5,530,240; 6,329,925;7,855,755; 7,626,749; 7,581,859; 7,446,650; 7,370,983; 7,338,177;7,274,501; 7,255,451; 7,195,381; 7,184,190; 5,668,663; 5,724,187 and/or6,690,268, and/or in U.S. patent applications, Ser. No. 12/091,525,filed Apr. 25, 2008, now U.S. Pat. No. 7,855,755; Ser. No. 11/226,628,filed Sep. 14, 2005 and published Mar. 23, 2006 as U.S. Publication No.US-2006-0061008; and/or Ser. No. 10/538,724, filed Jun. 13, 2005 andpublished Mar. 9, 2006 as U.S. Publication No. US-2006-0050018, whichare all hereby incorporated herein by reference in their entireties. Thedisplay is viewable through the reflective element when the display isactivated to display information. The display element may be any type ofdisplay element, such as a vacuum fluorescent (VF) display element, alight emitting diode (LED) display element, such as an organic lightemitting diode (OLED) or an inorganic light emitting diode, anelectroluminescent (EL) display element, a liquid crystal display (LCD)element, a video screen display element or backlit thin film transistor(TFT) display element or the like, and may be operable to displayvarious information (as discrete characters, icons or the like, or in amulti-pixel manner) to the driver of the vehicle, such as passenger sideinflatable restraint (PSIR) information, tire pressure status, and/orthe like. The mirror assembly and/or display may utilize aspectsdescribed in U.S. Pat. Nos. 7,184,190; 7,255,451; 7,446,924 and/or7,338,177, which are all hereby incorporated herein by reference intheir entireties. The thicknesses and materials of the coatings on thesubstrates of the reflective element may be selected to provide adesired color or tint to the mirror reflective element, such as a bluecolored reflector, such as is known in the art and such as described inU.S. Pat. Nos. 5,910,854; 6,420,036 and/or 7,274,501, which are herebyincorporated herein by reference in their entireties.

Changes and modifications in the specifically described embodiments maybe carried out without departing from the principles of the presentinvention, which is intended to be limited only by the scope of theappended claims as interpreted according to the principles of patentlaw.

1. A vehicular vision system, the vehicular vision system comprising: aplurality of cameras disposed at a vehicle equipped with the vehicularvision system, the plurality of cameras capturing image data as theequipped vehicle is driven along a traffic lane of a road; wherein theplurality of cameras comprises (i) a rear camera disposed at a rearportion of the equipped vehicle and having a field of view at leastrearward of the equipped vehicle and (ii) side cameras disposed atrespective side portions of the equipped vehicle, and wherein the sidecameras comprise (a) a left-side camera disposed at a left side portionof the equipped vehicle and having a field of view at least sideward andrearward of the equipped vehicle and (b) a right-side camera disposed ata right side portion of the equipped vehicle and having a field of viewat least sideward and rearward of the equipped vehicle; a processordisposed at the equipped vehicle; wherein image data captured by camerasof the plurality of cameras is provided to the processor; whereincaptured image data provided to the processor is processed to detect arearward-approaching vehicle that is approaching the equipped vehiclefrom rearward of the equipped vehicle and that is present in the fieldof view of at least one camera of the plurality of cameras; a videodisplay device disposed at the equipped vehicle, the video displaydevice comprising a video display screen that is viewable by a driver ofthe equipped vehicle; wherein the video display screen is operable todisplay video images derived from image data captured by at least onecamera of the plurality of cameras; wherein, as the equipped vehicle isdriven along the traffic lane of the road, and responsive to detectionof the rearward-approaching vehicle via processing of captured imagedata, the vehicular vision system determines a path of travel of thedetected rearward-approaching vehicle; wherein, as the equipped vehicleis driven along the traffic lane of the road, and responsive todetermining that the path of travel of the detected rearward-approachingvehicle is along a side traffic lane adjacent to the traffic lane alongwhich the equipped vehicle is driven, the vehicular vision system (i)displays at the video display screen video images derived at least fromimage data captured by the side camera that is at the side portion ofthe equipped vehicle that is adjacent to the side traffic lane and (ii)overlays a graphic overlay overlaying the displayed video images that isbased on the determined path of travel of the detectedrearward-approaching vehicle; and wherein the graphic overlay comprisesa partially transparent graphic overlay.
 2. The vehicular vision systemof claim 1, wherein the field of view of the left-side camera partiallyoverlaps the field of view of the rear camera, and wherein the field ofview of the right-side camera partially overlaps the field of view ofthe rear camera.
 3. The vehicular vision system of claim 1, wherein thegraphic overlay overlaying the displayed video images is indicative ofthe determined path of travel of the detected rearward-approachingvehicle.
 4. The vehicular vision system of claim 1, wherein thepartially transparent graphic overlay overlays a portion of thedisplayed video images, and wherein the video images displayed at thatportion are at least partially viewable by the driver of the equippedvehicle through the partially transparent graphic overlay.
 5. Thevehicular vision system of claim 1, comprising a non-imaging sensordisposed at the equipped vehicle and having a field of sensing at leastrearward of the equipped vehicle, the non-imaging sensor capturingsensor data as the equipped vehicle is driven along the traffic lane ofthe road.
 6. The vehicular vision system of claim 5, wherein thevehicular vision system, via processing of sensor data captured by thenon-imaging sensor, determines distance to the detectedrearward-approaching vehicle.
 7. The vehicular vision system of claim 5,wherein the vehicular vision system, via processing of sensor datacaptured by the non-imaging sensor, determines a change in distance tothe detected rearward-approaching vehicle.
 8. The vehicular visionsystem of claim 5, wherein the non-imaging sensor comprises a radarsensor.
 9. The vehicular vision system of claim 5, wherein thenon-imaging sensor comprises a lidar sensor.
 10. The vehicular visionsystem of claim 5, wherein the non-imaging sensor comprises a left-sidenon-imaging sensor accommodated in a left-side exterior rearview mirrorat the left side portion of the equipped vehicle and a right-sidenon-imaging sensor accommodated in a right-side exterior rearview mirrorat the right side portion of the equipped vehicle.
 11. The vehicularvision system of claim 1, wherein the graphic overlay comprises aplurality of rectangular overlays overlaying where the side traffic laneis displayed in the displayed video images.
 12. The vehicular visionsystem of claim 1, wherein, as the equipped vehicle is driven along thetraffic lane of the road, and responsive to determining that the path oftravel of the detected rearward-approaching vehicle is along a left-sideside traffic lane adjacent to the traffic lane along which the equippedvehicle is driven, the vehicular vision system displays at the videodisplay screen video images derived at least from image data captured bythe left-side camera.
 13. The vehicular vision system of claim 1,wherein, as the equipped vehicle is driven along the traffic lane of theroad, and responsive to determining that the path of travel of thedetected rearward-approaching vehicle is along a right-side side trafficlane adjacent to the traffic lane along which the equipped vehicle isdriven, the vehicular vision system displays at the video display screenvideo images derived at least from image data captured by the right-sidecamera.
 14. The vehicular vision system of claim 1, wherein theprocessor processes captured image data over multiple frames of capturedimage data to track the detected rearward-approaching vehicle as itapproaches the equipped vehicle.
 15. The vehicular vision system ofclaim 1, wherein the processor processes captured image data overmultiple frames of captured image data to track the detectedrearward-approaching vehicle as it overtakes the equipped vehicle. 16.The vehicular vision system of claim 1, wherein the left-side camera isdisposed at a left-side exterior rearview mirror at the left sideportion of the equipped vehicle, and wherein the right-side camera isdisposed at a right-side exterior rearview mirror at the right sideportion of the equipped vehicle.
 17. The vehicular vision system ofclaim 1, wherein the video display screen is disposed at an interiorrearview mirror assembly of the equipped vehicle.
 18. A vehicular visionsystem, the vehicular vision system comprising: a plurality of camerasdisposed at a vehicle equipped with the vehicular vision system, theplurality of cameras capturing image data as the equipped vehicle isdriven along a traffic lane of a road; wherein the plurality of camerascomprises (i) a rear camera disposed at a rear portion of the equippedvehicle and having a field of view at least rearward of the equippedvehicle and (ii) side cameras disposed at respective side portions ofthe equipped vehicle, and wherein the side cameras comprise (a) aleft-side camera disposed at a left-side exterior rearview mirror at aleft side portion of the equipped vehicle and having a field of view atleast sideward and rearward of the equipped vehicle and (b) a right-sidecamera disposed at a right-side exterior rearview mirror at a right sideportion of the equipped vehicle and having a field of view at leastsideward and rearward of the equipped vehicle; a processor disposed atthe equipped vehicle; wherein image data captured by cameras of theplurality of cameras is provided to the processor; wherein capturedimage data provided to the processor is processed to detect arearward-approaching vehicle that is approaching the equipped vehiclefrom rearward of the equipped vehicle and that is present in the fieldof view of at least one camera of the plurality of cameras; a videodisplay device disposed at an interior rearview mirror assembly of theequipped vehicle, the video display device comprising a video displayscreen that is viewable by a driver of the equipped vehicle; wherein thevideo display screen is operable to display video images derived fromimage data captured by at least one camera of the plurality of cameras;wherein, as the equipped vehicle is driven along the traffic lane of theroad, and responsive to detection of the rearward-approaching vehiclevia processing of captured image data, the vehicular vision systemdetermines a path of travel of the detected rearward-approachingvehicle; wherein, as the equipped vehicle is driven along the trafficlane of the road, and responsive to determining that the path of travelof the detected rearward-approaching vehicle is along a side trafficlane adjacent to the traffic lane along which the equipped vehicle isdriven, the vehicular vision system (i) displays at the video displayscreen video images derived at least from image data captured by theside camera that is at the side portion of the equipped vehicle that isadjacent to the side traffic lane and (ii) overlays a graphic overlayoverlaying the displayed video images that is based on the determinedpath of travel of the detected rearward-approaching vehicle; and whereinthe graphic overlay comprises a partially transparent graphic overlay.19. The vehicular vision system of claim 18, wherein the field of viewof the left-side camera partially overlaps the field of view of the rearcamera, and wherein the field of view of the right-side camera partiallyoverlaps the field of view of the rear camera.
 20. The vehicular visionsystem of claim 18, wherein the graphic overlay overlaying the displayedvideo images is indicative of the determined path of travel of thedetected rearward-approaching vehicle.
 21. The vehicular vision systemof claim 18, wherein the partially transparent graphic overlay overlaysa portion of the displayed video images, and wherein the video imagesdisplayed at that portion are at least partially viewable by the driverof the equipped vehicle through the partially transparent graphicoverlay.
 22. The vehicular vision system of claim 18, wherein thegraphic overlay comprises a plurality of rectangular overlays overlayingwhere the side traffic lane is displayed in the displayed video images.23. The vehicular vision system of claim 18, wherein, as the equippedvehicle is driven along the traffic lane of the road, and responsive todetermining that the path of travel of the detected rearward-approachingvehicle is along a left-side side traffic lane adjacent to the trafficlane along which the equipped vehicle is driven, the vehicular visionsystem displays at the video display screen video images derived atleast from image data captured by the left-side camera.
 24. Thevehicular vision system of claim 18, wherein, as the equipped vehicle isdriven along the traffic lane of the road, and responsive to determiningthat the path of travel of the detected rearward-approaching vehicle isalong a right-side side traffic lane adjacent to the traffic lane alongwhich the equipped vehicle is driven, the vehicular vision systemdisplays at the video display screen video images derived at least fromimage data captured by the right-side camera.
 25. The vehicular visionsystem of claim 18, wherein the processor processes captured image dataover multiple frames of captured image data to track the detectedrearward-approaching vehicle as it approaches the equipped vehicle. 26.The vehicular vision system of claim 18, wherein the processor processescaptured image data over multiple frames of captured image data to trackthe detected rearward-approaching vehicle as it overtakes the equippedvehicle.
 27. A vehicular vision system, the vehicular vision systemcomprising: a plurality of cameras disposed at a vehicle equipped withthe vehicular vision system, the plurality of cameras capturing imagedata as the equipped vehicle is driven along a traffic lane of a road;wherein the plurality of cameras comprises (i) a rear camera disposed ata rear portion of the equipped vehicle and having a field of view atleast rearward of the equipped vehicle and (ii) side cameras disposed atrespective side portions of the equipped vehicle, and wherein the sidecameras comprise (a) a left-side camera disposed at a left-side exteriorrearview mirror at a left side portion of the equipped vehicle andhaving a field of view at least sideward and rearward of the equippedvehicle and (b) a right-side camera disposed at a right-side exteriorrearview mirror at a right side portion of the equipped vehicle andhaving a field of view at least sideward and rearward of the equippedvehicle; a processor disposed at the equipped vehicle; wherein imagedata captured by cameras of the plurality of cameras is provided to theprocessor; wherein captured image data provided to the processor isprocessed to detect a rearward-approaching vehicle that is approachingthe equipped vehicle from rearward of the equipped vehicle and that ispresent in the field of view of at least one camera of the plurality ofcameras; a non-imaging sensor disposed at the equipped vehicle andhaving a field of sensing at least rearward of the equipped vehicle, thenon-imaging sensor capturing sensor data as the equipped vehicle isdriven along the traffic lane of the road; wherein the vehicular visionsystem, via processing of sensor data captured by the non-imagingsensor, determines distance to the detected rearward-approachingvehicle; a video display device disposed at the equipped vehicle, thevideo display device comprising a video display screen that is viewableby a driver of the equipped vehicle; wherein the video display screen isoperable to display video images derived from image data captured by atleast one camera of the plurality of cameras; wherein, as the equippedvehicle is driven along the traffic lane of the road, and responsive todetection of the rearward-approaching vehicle via processing of capturedimage data, the vehicular vision system determines a path of travel ofthe detected rearward-approaching vehicle; wherein, as the equippedvehicle is driven along the traffic lane of the road, and responsive todetermining that the path of travel of the detected rearward-approachingvehicle is along a side traffic lane adjacent to the traffic lane alongwhich the equipped vehicle is driven, the vehicular vision system (i)displays at the video display screen video images derived at least fromimage data captured by the side camera that is at the side portion ofthe equipped vehicle that is adjacent to the side traffic lane and (ii)overlays a graphic overlay overlaying the displayed video images that isbased on the determined path of travel of the detectedrearward-approaching vehicle; and wherein the graphic overlay comprisesa partially transparent graphic overlay.
 28. The vehicular vision systemof claim 27, wherein the field of view of the left-side camera partiallyoverlaps the field of view of the rear camera, and wherein the field ofview of the right-side camera partially overlaps the field of view ofthe rear camera.
 29. The vehicular vision system of claim 27, whereinthe graphic overlay overlaying the displayed video images is indicativeof the determined path of travel of the detected rearward-approachingvehicle.
 30. The vehicular vision system of claim 27, wherein thepartially transparent graphic overlay overlays a portion of thedisplayed video images, and wherein the video images displayed at thatportion are at least partially viewable by the driver of the equippedvehicle through the partially transparent graphic overlay.
 31. Thevehicular vision system of claim 27, wherein the non-imaging sensorcomprises a radar sensor.
 32. The vehicular vision system of claim 27,wherein the non-imaging sensor comprises a lidar sensor.
 33. Thevehicular vision system of claim 27, wherein the non-imaging sensorcomprises a left-side non-imaging sensor accommodated in the left-sideexterior rearview mirror at the left side portion of the equippedvehicle and a right-side non-imaging sensor accommodated in theright-side exterior rearview mirror at the right side portion of theequipped vehicle.
 34. The vehicular vision system of claim 27, whereinthe graphic overlay comprises a plurality of rectangular overlaysoverlaying where the side traffic lane is displayed in the displayedvideo images.
 35. The vehicular vision system of claim 27, wherein, asthe equipped vehicle is driven along the traffic lane of the road, andresponsive to determining that the path of travel of the detectedrearward-approaching vehicle is along a left-side side traffic laneadjacent to the traffic lane along which the equipped vehicle is driven,the vehicular vision system displays at the video display screen videoimages derived at least from image data captured by the left-sidecamera.
 36. The vehicular vision system of claim 35, wherein, as theequipped vehicle is driven along the traffic lane of the road, andresponsive to determining that the path of travel of the detectedrearward-approaching vehicle is along a right-side side traffic laneadjacent to the traffic lane along which the equipped vehicle is driven,the vehicular vision system displays at the video display screen videoimages derived at least from image data captured by the right-sidecamera.
 37. The vehicular vision system of claim 36, wherein theprocessor processes captured image data over multiple frames of capturedimage data to track the detected rearward-approaching vehicle as itapproaches the equipped vehicle.
 38. The vehicular vision system ofclaim 37, wherein the video display screen is disposed at an interiorrearview mirror assembly of the equipped vehicle.
 39. The vehicularvision system of claim 36, wherein the processor processes capturedimage data over multiple frames of captured image data to track thedetected rearward-approaching vehicle as it overtakes the equippedvehicle.